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Chapter  1  !  Introduction 


a.  Handbook  Purpose: 

V) 

This  handbook  is  designed  to  help  you  develop  and  use  good  metrics.  It  is 
intended  to  provide  you  with  sufficient  information  to  begin  developing 
metrics  for  your  own  objectives,  processes,  and  tasks,  and  to  steer  you  toward 
appropriate  actions  based  on  the  data  you  collect  It  should  be  viewed  as  a  road 
map  to  assist  you  in  arriving  at  meaningful  metrics  and  to  assist  in  continuous 
process  improvement  This  handbook  is  the  result  of  a  team  effort  by  people 
from  the  product  divisions,  test  centers,  and  other  command  organizations  who 
'  have  struggled  to  develop  their  own  metrics. 

Chapter  One  answers  your  most  basic  questions  about  metrics.  Chapter  Two 
provides  the  characteristics  of  good  metrics  and  discusses  what  metrics  are 
NOT  meant  to  be.  Chapter  Throe  provides  a  methodology  for  developing 
metrics  and  introduces  the  concept  of  the  metric  package.  Chapter  Four 
explains  performance  and  introduces  some  of  the  tools  used  in  analyzing 
processes  and  developing  metrics.  Chapter  Five  presents  some  guidelines  and 
Ideas  for  metrics  presentation.  Chapter  six  addresses  implementation-what  to 
do  with  metrics  once  you  have  them.  The  handbook  closes  with  three  appen¬ 
dices.  Appendix  A  is  a  brief  orientation  to  the  eight  basic  tools  introduced  in 
Chapter  Four.  Appendix  B  is  a  glossary  of  terms  helpful  in  understanding 
metrics.  Appendix  C  is  a  resource  of  references  that  will  provide  you  with  more 
help.  _ 

A  second  volume  of  this  handbook  is  planned  for  later  publication.  This  will 
contain  the  Command  Standard  Metrics  and  Big  Picture  Metrics.  It  will  be 
maintained  and  published  out  of  the  command  headquarters. 

o.  Metrics  -  Measurement  You  Can  Uaa: 

Metrics  are  nothing  more  than  meaningful  measures.  For  a  measure  to  be 
meaningful,  however,  it  must  present  data  that  allow  us  to  take  action.  It  must 
be  customer  oriented  and  support  the  meeting  of  our  organizational  goals  and 
objectives.  Metrics  foster  process  understanding  and  motivate  acdon  to 
continually  improve  the  way  we  do  business.  Tnis  is  distinguished  from 
measurement,  in  that,  measurement  does  not  necessarily  result  in  process 
improvement  Good  metrics  always  will. 

Cl.  Metrics  and  Command  Strategic  Planning: 

Metrics  also  play  an  integral  role  in  linking  organizational  processes  to  the 
achievement  of  tne  Command  Corporate  Plan.  They  are  the  diagnostics  which 
show  progress  in  meeting  corporate  goals  and  objectives.  As  shown  in  Fig  1- 
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1  .corporate  planning  begins  with  a  statement  of  the  Command  Mission  and  its 
Vision  of  the  future.  A  strategic  analysis  of  major  internal  and  external 
influences  identifies  key  issues  which  must  be  addressed  to  achieve  that  long¬ 
term  Vision.  The  analysis  and  key  issues  define  where  current  capabilities  and 
future  organizational  requirements  are  not  aligned.  The  Strategic  Plan  culmi¬ 
nates  in  the  identification  of  goals  which  specifically  address  those  issues. 

Goals  and  their  subordinate  near-term  objectives  chan  the  appropriate  course 
of  action.  Those  objectives,  in  many  cases,  include  the  improvement  of 
processes  and  tasks  which  involve  command  organizations  and  personnel. 
You  should  be  able  to  align  selected  metrics  to  organizational  objectives  and 
identify  which  processes  are  targeted  for  improvement  through  their  applica¬ 
tion.  Organizational  objectives,  and  processes,  and  metrics  which  are  devel¬ 
oped  to  suppon  them  make  up  the  organizations'  business  plan. 


Figure  1-1:  Corporate  Planning  Framework 
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Good  metrics  will  gauge  progress  in  reaching  the  processes'  desired  outcomes. 
This  will  lead  to  accomplishment  of  organizational  objectives.  This  will 
consequently  result  in  achieving  the  Command  Goals  and  Vision. 


t.  Why  Metrics?: 

Metrics  facilitate  quality  improvement.  Metrics  help  us  understand  our  pro¬ 
cesses  and  their  capabilities  so  we  can  continually  improve  them.  As  Belasco 
points  out  "what  gets  measured  gets  empowered  and  produced."'  Good  metrics 
will  help  us  "face  reality,  communicate  it,  and  deal  with  it."1 


f.  Who  Netds  Mttrlcs?: 

Metrics  can  apply  to  any  individual  or  any  organization  responsible  for  a  task, 
activity,  system,  or  process.  They  can  be  developed  and  integrated  into  any 
organization's  long-range  planning  process  at  any  level.  Metrics  can  be  applied 
to  any  ongoing  or  recurring  task,  activity,  system,  or  process.  One  time  tasks 
and  projects  are  still  measurable,  but  the  benefits  of  measuring  may  not  be 
worm  the  effort 


'  JtflM  A.  Taidirn  i)»  Elephent  lo  D«iee.  New  York,  NY:  Crown.  1990,  p  1  SI 

1  Perry  L  Jotawon,  gfiQiailSBOL New  York,  NY:  Kerper  A  Row  Publuhen,  1989,  p.  1 1 1. 
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Chapter  2  !  Fundamentals 


a.  What  Metrics  Ara: 

As  our  Commander  has  stated,  metrics  are  measures  of  how  we  are  doing3.  That 
implies  several  things.  First  and  foremost,  since  we  are  a  service  organization, 
our  metrics  must  be  founded  in  a  customer  orientation.  Second,  a  metric  must 
communicate  the  health  of  a  process.  Third,  the  metric  must  distinguish 
"health"  from  sickness.  To  know  what  health  means,  we  must  know  where  we 
are  now  and  where  we  want  to  go.  Fourth,  a  time  dimension  is  necessary  for 
true  metrics. 

For  purposes  of  this  handbook,  a  metric  will  be  defined  as  a  measurement  made 
over  time,  which  communicates  vital  information  about  the  quality  of  a 
process,  activity,  or  resource. 

Measurement  is  a  means  to  an  end:  continuous  improvement  of  the  way  we  do 
business. 


b.  Attributes  of  a  Good  Metric: 

The  following  are  the  basic  characteristics  of  a  good  metric:4 

1.  It  is  accepted  as  meaningful  to  the  customer. 

2.  It  tells  how  well  organizational  goals  and  objectives  are  being  met  through 
processes  and  tasks. 

3.  It  is  simple,  understandable,  logical  and  repeatable. 

4.  It  shows  a  trend. 

5.  It  is  unambiguously  defined. 

6.  Its  data  is  economical  to  collect. 

7.  It  is  timely. 

BUT  MOST  IMPORTANTLY: 

8.  It  drives  the  "appropriate  action." 

*  R  or  aid  W.  Yatei,  AFSC/CC.  Letter  to  HQ  AFSC  DCS,  ACS,  and  SSO»,  20  Aug  90 

*  Primarily  derived  from  a  handout  from  the  26  Sep  90  AFSC  Commander'i  Suff  Meeting 
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e.  What  Metric*  are  Not: 

Metrics  are  not: 

1 .  Charts.  Charts  may  graphically  display  the  results  of  a  metric .  But  the  chan 
itself  is  not  the  metric. 

2.  Schedules.  Some  forms  of  schedule  can  lead  to  good  metrics,  but  usually 
schedules  do  not  provide  information  that  by  itself  will  lead  to  process 
improvement. 

3.  Goals,  Objectives,  Strategies,  Plans,  Missions,  or  Guiding  Principles. 
Most  of  these  can  be  measured,  but  as  we  stated  before,  the  metric  is  not  by 
itself  the  end.  It  is  a  means  to  an  end  -  to  achieving  objectives  and  goals 
through  process  improvement. 

4.  Counts  of  activity.  Counts  of  activity  can  result  in  metrics,  but  just  because 
you  have  statistics  does  not  mean  that  you  have  a  measure  that  will  drive 
appropriate  action. 

5.  Snapshots  or  one-time  status  measures  as  displayed  in  pie  charts,  stoplight 
charts,  etc.  These  convey  little  trend  information.  Comparison  of  status 
over  time  can  be  a  metric,  but  it  tends  to  be  very  top  level  and  does  not 
provide  much  understanding  of  the  process. 


d.  Metrics  and  Process  Variation: 


Ail  processes  exhibit  some  variation.  Continuous  improvement  of  any  process 
means  pursuing  uniformity  of  output,  that  is,  reduction  of  variation.  If  vou  do 
not  know  the  variation  of  a  process  and  you  change  the  process,  you  risk  having 
no  effect  or  making  the  process  worse.  You  will  be  "tampering"  with  the 
process  rather  than  systematically  making  improvements.  The  discussion  of 
normal  and  abnormal  variation  and  the  use  of  histograms  and  control  chans 
described  in  the  Appendix  A  provide  useful  information  for  interpreting 
metrics. 
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Chapter  3:  Development  Process 


a.  General: 

The  previous  chapters  of  this  Handbook  were  oriented  toward  providing  you 
with  the  theoretical  framework  of  metrics.  This  chapter  deals  with  the  nuts 
and  bolts  of  developing  metrics.  It  describes  the  concept  of  a  physical 
"Metric  Package,"  and  provides  a  step-by-step  process  for  developing 
metrics. 

As  mentioned  previously,  metrics  are  measurements  you  can  use.  They  are 
a  subset  of  the  variety  of  measurements  we  can  make.  Some  measurements 
of  data  have  the  potential  to  become  metrics,  but  not  all. 

Ultimately,  two  criteria  must  be  satisfied  for  a  measurement  to  be  an 
effective  metric.  As  mentioned  before,  a  metric  must  present  us  with  data 
we  can  use.  It  should  show  a  status  over  time.  Only  trend  data  has  the 
tential  to  be  evaluated  to  the  degree  needed  in  order  to  establish  actions  to 
taken.  It  should  further  present  us  with  information  which  allows  us  to 
take  some  initial  action,  Second,  a  metric  must  directly  support  corporate 
goals  or  objectives  because  it  is  built  from  the  strategic  and  business  plans. 
All  efforts  to  evaluate  yourcurrent  situation  and  steps  taken  to  improve  your 
processes  will  be  in  vain  unless  the  end  result  is  the  advancement  of  the 
organization  toward  successfully  meeting  corporate  goals.  Both  of  these 
attributes  are  further  elaborated  on  in  Chapter  Five.  The  importance  of  each 
will  become  apparent  once  you've  begun  to  make  changes  and  are  well  into 
the  implementation  phase. 

b.  The  "Metric  Package": 


A  metric  consists  of  the 
following  three  basic  el¬ 
ements:  the  operational 
definition,  the  actual 
measurement  and  re¬ 
cording  of  data,  and  the 
metric  presentation.  To¬ 
gether  these  elements 
are  called  the  metric 
package.  This  is  illus¬ 
trated  in  Figure  3- 1 .  The 
operational  definition  is 
the  precise  explanation 
of  the  process  being 
measured.  r.  ne  mea¬ 
surement  and  data  col¬ 
lection  is  the  translation 
of  data  from  the  process  into  understandable  and  useful  information.  The 
metric  presentation  is  the  metric's  communication  link  to  the  process 
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customer  and  process  manager. 

The  first  element  of  the  metric  package  is  the  operational  definition.  According 
to  some  "there  is  nothing  more  important  for  the  transaction  of  business  than 
the  use  of  operational  definitions."’  The  operational  definition  is  the  who, 
what,  when,  where,  and  how  of  the  metric.  It  is  customer  oriented  and  accepted. 

It  must  be  detailed  enough  to  allow  valid,  repeatable,  and  consistent  measurement 
to  be  made  over  time  and  is  the  key  to  internal  communication  for  process 
understanding  which  is  a  vital  step  if  we  hope  to  have  improvement.  The 
amount  of  detail  required  will  vary  from  metric  to  metric,  but  at  a  minimum,  \ 

it  should  consist  of  the  following  elements: 

•  An  unambiguous  description  of  the  metric 

•  The  population  that  the  metric  will  include 

•  The  frequency  of  measurement 

•  The  source  of  the  data 

•  Any  equations  required  in  doing  the  measurement 

•  Precise  definition  of  key  terms 

•  A  description  of  the  graphic  presentation  that  will  eventually  be  used  to 
display  the  data 

•  The  customer  of  the  metric 

•  The  accountable  process  owner 

•  The  desired  outcome  expressed  in  teims  of  a  positive  or  negative  trend  (not 
a  numerical  goal) 

•  The  link  between  the  process  being  measured,  your  organization's  strategic 
plan,  and  the  command  goals. 

The  second  element  is  the  actual  measurement  of  the  process.  This  is  the  step 
of  collecting  and  recording  the  data.  This  step  is  the  hem  of  the  metric  concept 
and  serves  as  the  translation  of  data  from  the  process  being  measured  to 
meaningful  information  that  will  be  the  basis  of  your  process  improvement 
activities. 

The  third  element  of  the  metric  package,  the  metric  presentation,  has  two  parts.  The 
first  part  is  the  metric  descriptor  and  the  second  part  is  the  graphic  presentation  of 
the  data.  The  descriptor  and  presentation  are  actually  planned  when  developing  the 
operational  definition,  before  most  of  the  data  is  collected.  The  amount  of  detail 
required  in  the  descriptor  will  vary  with  the  purpose  of  the  presentation.  At  a 
minimum,  it  will  include  a  clear  definition  of  the  metric,  the  source  of  the  data,  the 
population  that  the  metric  will  include,  the  owner  of  the  process  the  metric 
measures,  and  identification  of  the  command  goal  that  this  process  is  linked  to.  It 
should  contain  enough  information  to  clearly  communicate  the  information 
contained  in  the  second  part  of  the  presentation.  The  second  part  of  the  metric 
presentation,  the  chart,  is  often  mistakenly  referred  to  as  the  metric.  Together  the 
descriptor  and  the  chart  should  provide  enough  information  to  clearly  communi¬ 
cate  the  metric.  These  are  further  discussed  in  Chapter  Five. 

*W.  Edwards  Demming,  Out  of  the  Crisis.  Cambridge.  MA:  MIT  CAES,  1986,  p.  276. 
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c.  Tho  Stops  to  Motrlc  Dovolopmont: 

This  section  provides  the  steps  for  formulating  the  metric  package  and 
measuring  associated  process  improvement.  Keep  in  mind  that  these  steps  may 
require  tailoring  before  they  can  be  applied  to  your  particular  situation.  These 
steps  are  pictured  with  an  example  in  Figure  3-2, 

STEF  I.  IDENTIFY  YOUR  PURPOSE 

It  is  important  to  first  align  your  purpose  with  your  organization's  mission, 
vision,  goals,  and  objectives.  These  should  be  inextricably  linked  to  meeting 
customer  needs  and  serve  as  a  foundation  for  accomplishing  and  sustaining 
continuous,  measurable  improvement. 

STEP  H.  DEVELOP  YOUR  OPERATIONAL  DEFINITION  STARTING 
WITH  YOUR  CUSTOMER 

Define  the  who,  what,  when,  why  and  how  of  this  metric  in  sufficient  detail  to 
permit  consistent,  repeatable  and  valid  measurement  to  take  place.  The 
operational  definition  starts  with  an  understanding  of  your  customers'  expec¬ 
tations.  You  then  "operationalize"  theexpectation(s)  bydefmingcharacterisuc(s) 
of  the  product,  service,  or  process  which  are  internally  measurable  and  which, 
if  improved,  w  ould  better  satisfy  your  customers'  expectations.  This  is  actually 
an  iterative  process  involving  Steps  II- VII.  This  is  the  first  element  of  your 
metric  package. 

STEP  in.  IDENTIFY  AND  EXAMINE  EXISTING  MEASUREMENT 
SYSTEMS 

Once  the  link  to  objectives  and  goals  has  been  established,  it  is  essential  to 
determine  if  existing  metrics  or  other  measurement  systems  exist  that  satisfy 
your  requirements.  Don't  "reinvent  the  wheel."  Use  existing  process  mea¬ 
surements  when  they  exist. 

STEP  IV.  GENERATE  NEW  METRICS  IF  EXISTING  METRICS  ARE 
INADEQUATE 

Most  measurements  used  in  the  past  were  not  process  oriented.  They  were 
results  indicators  related  to  final  outputs,  products  or  services  for  external 
customers,  With  metrics,  the  focus  is  on  now  processes  are  performing  in 
making  these  final  outputs.  We  are  interested  in  those  upstream  process 
measures  which  drive  the  final  outcome  and  are  the  key  to  making  process 
improvements.  The  assumption  is:  if  process  performance  is  monitored  and 
improved,  the  quality  of  the  products  and  services  will  improve. 
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STEP  V.  RATE  YOUR  METRIC  AGAINST  THE  "EIGHT  ATTRIBUTES 
OF  A  GOOD  METRIC" 

Refer  to  the  attributeslistedinChapterTwo.  If  you  feel  yourmetric  sufficiently 
satisfies  these  criteria  for  a  good  metric,  go  on  to  Step  VI.  If  not,  return  to  Step 
II  and  correct  the  deficiencies. 

STEP  VI.  SELECT  APPROPRIATE  MEASUREMENT  TOOLS 

Select  the  proper  tool  for  analyzing  and  displaying  your  data.  The  "Eight  Basic 
Tools"  identified  in  Figure  4-1  are  those  most  commonly  used.  Other  statistical 
and  non-stadsdcal  tools  may  be  more  appropriate  for  your  application.  Use 
whatever  you  feel  is  best.  The  tools  will  be  discussed  further  in  Chapter  Four 
and  Appendix  C. 

STEP  VII.  BASELINE  YOUR  PROCESS 

Start  acquiring  metric  data.  This  serves  as  a  baseline  for  determining  the 
capability  of  your  process.  Ask  if  the  data,  is  accumulated  over  time  and 
adequately  measures  the  important  characteristics  of  your  process.  If  the 
answer  is  uncertain,  examine  other  possibilities.  And  if  you  change  your 
metric,  remember  to  coordinate  it  with  your  customer  again. 

STEP  VIII.  COLLECT  AND  ANALYZE  METRICS  DATA  OVER  TIME 

Continue  aggregating  metric  data  over  time.  Examine  trends.  Special  and/or 
common  cause  effects  on  the  data  should  be  investigated  and  assigned. 
Compare  the  data  to  interim  performance  levels.  This  is  the  second  clement  of 
your  metric  package. 

STEP  DC.  FINALIZE  TOE  METRIC  PRESENTATION 

Based  on  the  results  of  the  previous  steps,  you  are  finally  ready  to  present  the 
metric  externally.  The  descriptor  will  provide  enough  information  to  com¬ 
municate  the  appropriate  details  of  the  metric  to  your  customer.  The  appropri¬ 
ate  level  of  detail  should  be  determined  by  discussion  with  the  customer.  This 
information  should  be  an  abbreviation  of  the  key  elements  of  the  operational 
definition.  The  graphic  presentation  clearly  and  concisely  communicates  how 
you  are  performing,  This  is  the  third  element  of  your  metric  package. 
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STEP  X.  INITIATE  PROCESS  IMPROVEMENT  ACTIVITIES 


Initiate  process  improvement  activities  in  conjunction  with  the  key  process 
owners.  Chapter  Six  provides  more  guidance  in  this  area.  Once  improvements 
have  been  implemented,  the  process  above  may  start  over  or  it  may  pick  up 
again  at  almost  any  step.  Remember,  continuous  improvement  requires 
continuous  effort  THIS  STEP  IN  THE  DEVELOPMENT  PROCESS  IS  THE 
MOST  CRITICAL  FOR  YOUR  IMPROVEMENT  EFFORTS  TO  BECOME 
A  REALITY.  Remember  that  metrics  are  just  a  means  to  an  end!  That  end  is 
continuous  process  improvement. 
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Metric  Development  Process  Practical  Example 


I. 


II. 


in. 


IV. 


v. 


Figure  3-2:  Metric  Development  Process 
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(Continued) 


Metric  Descriptor  Chart 

Description: 

An  Averaged  Customer 

Satisfaction  Index  Using 

A  Closed  Scale  From  1 

(Very  Dissatisfied)  To  6 
(Very  Satisfied)  In  Five 
Discrete  Areas  Of 

Program  Execution, 
Program  Cost,  Product 

Acceptability,  Working 
Rel'ns,,  And  Optional 

Topics.  All  ACQ  Prgms 

To  Usa  User  Coordinated 

Customer  Satisfaction 
Asssssmant  Reports 
(CSARS)  At  Least 
Semi-Annually  Which 

Serve  As  The  Data 

Source  For  This  Metric. 

Desired 

Outcome: 

Improve  Customer 
Satisfaction  Rating  All  5 
Categories  As  An 

Average 

Linkage  to 
Strategic  Plan: 

Meet  User's  Needs 

Accountable 

Process 

Owners: 

Pgm  Mgrs/  SPO  Directors 

+ 


Metric  Graphs  (see  IX,  Above) 

I 


Figure  3*2  Metric  Development  Process 
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X. 


(Continued) 

X. 


\ 

Initiate  Process 
Improvement  Activities 


(End  Of  Example) 


i 


Ongoing  Efforts  Underway,  Scores  Of 
1  -2-3  Result  In  Further  Study  By  Specific 
Program  Manager 


Trend  Assessment  In  1  Or  2  More 
Measurement  Periods 


Good  Trends  Thus  Far  In  Program 
Execution,  Product  Acceptability,  And 
Optional  Topics 


Product  Division  &  Users  Still  Learning 
How  To  Use  CSAR's 


Trend  Is  Improving 


Color  Assessment:  Yellow 


Reason:  Customers  Are  Not  As  Satisfied 
As  Desired 


Figure  3-2:  Metric  Development  Process 
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Chapter  4i  Introduction  to  TQM  Tools 


a.  Procass  Performance: 


This  chapter  introduces  process  performance  and  several  common  statistical 
tools  that  will  be  useful  in  developing  metrics. 

Process  performance  means  how  and  how  well  a  process  works.  Since  what  the 
customer  wants  defines  your  standard  of  quality,  you  will  analyze  the  process 
in  terms  of  its  capability  to  meet  customer  needs  and  expectations.  The  tools 
in  Appendix  A  will  help  you  interpret  the  data  you  collect.  These  tools  are  a 
bit  luce  x-rays  used  by  doctors.  They  provide  pictures  which  enable  you  to 
understand  now  your  process  is  currently  performing. 

Depicting  and  understanding  process  variation  using  metrics  is  key  to  sys¬ 
tematically  improving  processes.  Why  study  variation?  It  is  the  cause  of  most 
unsatisfactory  performance  in  the  government  and  industry  today.  Variation 
adds  unnecessary  problems  to  any  job.  Some  of  the  consequences  often 
associated  with  variation  in  process  performance  are: 

•  Unpredictable  performance 

•  Additional  reviews  and  inspections  to  weed  out  undesirable  results 
«  Avoidable  rework  and/or  scrap 

•  Schedule  delays 

•  Ixjwer  productivity 

•  Lower  reliability 

•  Higher  costs 

•  Customer  dissatisfaction  (which,  by  definition,  means  poor  quality) 

There  are  two  types  of  variation:  abnormal  variation  and  normal  variation. 

Abnormal  variation  results  from  "special  causes,"  that  is,  something  unusual 
has  happened  to  the  process.  For  example,  errors  have  increased  because  a 
manager  assigned  a  new  and  untrained  employee  to  a  critical  process.  Special 
causes  can  usually  be  eliminated  by  people  close  to  the  process  and  without 
changing  the  process. 

Normal  variation  is  the  routine  fluctuation  of  process  performance  due  to  the 
interaction  of  the  process  components:  manpower,  machines,  work  methods, 
materials  and  the  work  environment.  Over  time,  process  indicator  data  may  go 
up  or  down,  but  you  can  predict  with  a  reasonable  degree  of  certainty  what  the 
range  of  variation  will  be.  Normal  variation  occurs  because  of  "common 
causes."  These  may  be  the  level  of  training  of  all  employees  in  a  process,  the 
reliability  of  equipment  and  methods  normally  used,  or  simply  minor  random 
events. 

Special  causes  account  for  only  about  15  percent  of  the  variation  in  a  process, 
and  common  causes  for  85  percent.  Process  performance  metrics  help  manag¬ 
ers  focus  their  attention  on  both  special  and  common  causes  of  variation.  Fires 
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(special  causes)  still  get  put  out,  but  now  continuous  improvement  (reducing 
die  effects  of  common  causes  of  process  variation)  gets  emphasized. 


•  What  is  the  total  variation  in  the  process?  Histograms  will  help  here. 

•  What  is  the  long-term  predictability  of  the  process?  Run  charts  and 
control  charts  will  provide  the  answer. 

•  What  is  the  short-term  stability  of  the  process?  Control  charts  (of  range 
values)  will  reveal  this. 

•  Is  the  variation  normal  or  abnormal?  Histograms  and  control  charts  can 
help  distinguish  between  these  two  types  of  variation. 


b.  Tools: 

The  same  statistical  tools  which  support  effective  process  control  can  be 
useful  in  developing  good  metrics.  You  must  decide  which  statistical 
approaches  are  appropriate  for  your  applications. 

The  tools  most  commonly  mentioned  in  TQM  literature  are  control  chans, 
run  charts,  flow  charts,  cause  and  effect  (or  fishbone)  diagrams,  histograms, 
Pareto  diagrams,  scatter  diagrams,  and  check  sheets6. 


Process  variation  is  often  disguised.  Therefore,  you  need  to  look  at  your  data 
and  your  metrics  on  the  performance  of  your  process  from  several  perspectives 
to  allow  the  process  to  "tell"  you  about  its  variation.  The  questions  you  want 
the  process  to  answer  are: 


'Memory  Jogger,  Walton,  Boot  Allen,  ScheTkenbach,  Kiemele,  et  tl. 
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Figure  4*1:  Eight  Useful  Tools 

Above  (Figure  4- 1 )  is  an  illustration  of  these  eight  tools.  Of  these  eight,  run 
chans  and  control  charts  most  readily  lend  themselves  to  the  display  of  good 
metrics.  This  is  because  they  best  show  process  performance  information 
over  time.  There  are  many  more  tools,  available  to  the  interested  user.  Table 
4-1  is  presented  to  help  you  determine  which  of  this  small  selection  of  tools 
may  best  apply  to  your  situation.  The  remaining  six  tools  are  oriented  toward 
problem  solving.  Those  who  understand  the  process  arc  in  the  best  position 
to  determine  which  statistical  techniques  to  apply.  It  is  suggested  that  those 
using  a  tool  understand  it  before  they  use  it.  Many  courses  and  books  are 
available  as  indicated  in  Appendix  C.  A  good  place  to  get  assistance  is  your 
organization's  TQM  office. 
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Table  4-1:  Overview  of  TQM  Took 


Table  4-1:  Overview  of  TQM  Tools 
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Chapter  5  !  Presentation 


This  chapter  provides  guidance  for  effectively  communicating  process 
performance  using  metrics. 

There  are  two  basic  elements  which  make  up  the  Metric  Presentation 
Package.  First,  a  metric  descriptor  should  be  provided  which  defines  your 
metric,  and  provides  other  essential  information  about  the  metric.  The 
"descriptor"  is  displayed  in  Figure  5-1  below. 

It  is  strongly  recommended  that  you  formulate  the  operational  definition 
as  specifically  as  possible  prior  to  formulating  your  metric  descriptor. 
Spending  as  much  time  as  is  warranted  on  both  will  pay  rich  dividends  down 
stream  in  terms  of  reduced  confusion  and  accuracy. 


Matrlc  Title 


Description: 

Briefly  define  your  metric  along  with  the  population  to  be  measured  and  the 
source  of  your  data.  These  and  other  items  of  information  on  your  metric 
should  be  contained  in  your  metric  operational  definition 


Deelred  Outcome: 

Define  the  outcome  in  terms  of  improved  processes.  Do  not  use  numerical 
goals. 

Linkage  to  Strategic  Pian: 

Identify  one  or  more  of  your  organization's  objectives  or  goals  that  are 
addressed  by  tracking  this  metric.  This  linkage  to  your  organization's 
business  plan  is  essential;  metrics  for  metrics'  sake  are  unwarranted. 

Procaas  Ownar: 

Identify  the  principal  individual  or  organization  who  can  initiate  and  sustain 
process  improvement. 


Figure  5-1:  Metric  Descriptor 
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The  second  element  of  a  Metric  Presentation  Package  is  a  graphic  illustration 
of  how  the  metric  tracks  over  time.  The  illustration  should  be  simple  to  interpret 
and  will  generally  be  either  a  run  chart  or  a  control  chart.  Both  were  introduced 
in  Chapter  Four  and  are  explained  in  more  detail  in  Appendix  A. 

In  addition  to  the  guidance  presented  in  Appendix  A,  a  few  other  helpful  hints 
are  appropriate.  Insure  your  graph  is  titled,  and  both  axes  are  labeled  as  well. 
Time  is  usually  represented  in  the  horizontal  or  x-axis,  and  your  metric,  as 
summarized  in  the  descriptor  chart,  is  labeled  on  the  vertical,  or  y-axis.  Use  a 
common  marker  for  each  data  string  such  as  an  x  or  o.  It  may  be  appropriate 
to  use  an  upward  or  downward  arrow  to  indicate  your  preference  for  raising  or 
lowering  the  metric  overall  in  time.  For  control  charts,  plot  the  center  line  and 
upper  and  lower  control  limits,  Your  upper  and  lower  control  limits  are 
statistically  calculated  based  on  your  process  capability.  They  are  never 
defined  from  customer  specifications  or  needs  I  And  in  most  cases,  both  your 
metric  descriptor  and  graphics  should  be  printed  horizontally  to  make  them 
easier  to  read. 

Figures  5-2  and  5-3  provide  an  example  of  a  complete  Metric  Presentation 
Package.  Figure  5-2  is  the  descriptor,  Figure  5-3  is  its  accompanying  graphic. 


Progam  Office  Overhead:  Metric  Description 

Reduce  Overhead 

Description: 

Avaraga  Percent Overheed 

Expraaaad  Ai  A  Function  01  Total 

Program  Coat.  All  Prograhu  Are 

Counted  Within  Tha  Division  That 

Cuirantly  Uaa  3800, 3400,  Or 

3080  Fundi,  Tha  Data  Will  Ba 

Obtained  From  Each  Program'a 

Current  FINPIAN.  Element!  Of 

Overheed  Indude 
FFRDC/TEMMETA.TDY, 

Overtime,  Suppllea,  Equipment, 

Training,  And  Aueiamenla . 

Deified  Outoemei 

A  DiviHonal  Trend  Which 

Indiciiei  A  Net  Average 

Reduction  Of  Overhead  Over 

Tima 

Untug*  to  Strategic  Rian: 

Matt  Uaer‘1  Need*  Through 

Reduced  Cocts 

Rrooasa  Own**: 

Program  Director 

Figure  5-2:  Sample  Metric  Descriptor 
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Figure  5>3:  Sample  Metric  Graphic 


Finally,  a  comment  on  "Stop  Light"  charts  is  appropriate.  Management  may 
wish  to  assign  a  color  rating  or  color-code  for  each  measurement  period.  An 
example  of  a  color  assessment  scheme  is  provided  in  Figure  5-5.  The  assign¬ 
ment  of  a  specific  color  may  be  predicated  on  the  trend  of  your  metric  over  time. 
An  illustration  of  how  a  Stop  Light  chart  could  be  integrated  into  a  metric  graph 
is  illustrated  below  in  Figure  5-4: 


iM—iwm;  (3D  ED  Q  (S  E  0 


Figure  5*4:  Sample  Stoplight  Assessment 


Establish  your  assessment  criteria  early-on,  and  stick  with  it  over  time.  Avoid 
changing  your  rules  at  mid-stream.  One  possible  set  of  assessment  criteria 
could  be  as  simple  as: 


Green  Process  in  control  and  capable 
Yellow  Process  in  control,  not  capable 
m  Process  out  of  control 


Figure  5-5:  Sample  Stoplight  Criteria 
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The  point  to  remember  is  to  establish  your  targets  and  threshold  after  careful 
consideration  of  your  customer's  needs  or  specifications  AND  an  understand¬ 
ing  of  what  your  process  is  capable  of  accomplishing.  Your  targets  or 
thresholds,  if  numerically  defined,  should  not  be  arbitrarily  established.  If  a 
color  code  is  also  required,  establish  your  color-coding  criteria  with  customer 
agreement,  and  maintain  consistency  in  this  criteria  over  time. 
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Chapter  6:  Implementing  Metrics  for 

Process  Improvement 


a.  Introduction: 

This  chapter  deals  with  the  task  of  using  data  and  metrics  to  systematically 
target  and  implement  process  improvements,  and  to  then  monitor  ana 
evaluate  the  effect  of  these  improvements.  Now  that  you  have  gathered  data 
and  presented  it  in  a  format  you  are  comfortable  with,  you  are  ready  to  take 
the  next,  and  most  important,  step  in  the  metrics  process.  This  is  the  step  of 
actually  using  the  data  to  effect  change  within  your  organization. 

To  an  extent,  implementation  began  informally  when  your  leadership  agreed 
with  customer  and  subordinate  on  what  would  be  measured.  Tacit  approval 
was  then  given  for  eventually  making  changes  based  on  the  metrics  data. 

The  ability  to  implement  change  based  on  accumulated  measured  data  is  the 
distinguishing  factor  between  taking  a  measurement  and  having  a  metric. 
Only  after  exercising  the  "appropriate  action"  step  of  implementation  have 
you  successfully  developed  true  metrics  and  taken  full  advantage  of  them. 


b.  Evaluation: 

As  mentioned  earlier,  every  metric  package  should  include  a  desired 
outcome.  A  fundamental  step  in  implementation  is  an  evaluation  of  where 
you  stand  -  with  respect  to  meeting  your  customer's  needs,  with  respect  to 
corporate  goals,  and  with  respect  to  any  appropriate  benchmark  or  compe¬ 
tition.  As  a  minimum,  this  evaluation  should  result  in  a  clear  understanding 
of  how  your  processes  are  achieving  customer  satisfaction. 

A  formal  approach  to  the  evaluation  may  take  place  using  Control  Charts.  In 
this  case,  Upper  Control  Limits  (UCLs)  and  Lower  Control  Limits  (LCLs) 
are  used  to  evaluate  whether  or  not  a  process  is  in  control.  The  formal 
approach  need  not  be  used,  however.  A  simpler  evaluation  may  adequately 
lead  to  the  "appropriate  action."  The  point  is  that  an  evaluation  need  not  be 
overly  complicated. 

An  important  tool  in  making  your  evaluation  will  be  your  customer's 
perspectives  and  feedback.  As  a  Total  Quality  organization,  the  value  of 
customer  satisfaction  is  paramount,  and  cannot  be  over-emphasized.  It  is 
vital  to  consider  the  customer's  input  in  any  assessment  of  performance  and 
process  changes. 

Your  evaluation  may  also  be  based  on  an  assessment  of  your  position  relative 
to  that  of  others,  often  your  competition.  Such  an  assessment  can  be 
extremely  beneficial,  but  only  if  a  positive  attitude  toward  benchmarking 
exists  in  the  organization.  In  the  past,  the  tendency  of  military  organizations 
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to  perceive  comparisons  with  a  "report-card  mentality"  was  exercised.  But  a 
competitive  evaluation  may  foster  improvement  by  demonstrating  that  a 
particular  process  can  be  further  improved,  and  suggesting  possible  approaches 
to  make  improvements. 


c.  Basis  for  Action: 

The  need  to  improve  is  fundamentally  important  to  any  organization,  This  is 
true  regardless  of  your  motivation  and  whether  you  are  ahead  of  or  behind  your 
desired  outcome.  Without  improvement,  there  is  stagnancy  at  best.  The 
inevitable  result  is  failure. 

There  are  many  sources  from  which  to  draw  upon  in  determining  an  effective 
action  to  improve  a  process.  Five  valuable  sources  are  the  process  itself,  your 
customer,  your  people,  your  competition,  and  your  competition's  customers.  A 
more  general  source  is  anyone  reputed  to  be  the  best  at  what  they  do  or  to  have 
established  the  "best  practices"  in  so  doing. 

If  your  metrics  are  effective,  you'll  never  have  to  ask  "Now  What?"  or  "So 
What?"  after  an  evaluation  of  your  data.  Good  metrics  should  enable  your 
process  to  provide  at  least  an  initial  indication  of  where  to  focus  your  energies 
toward  uncovering  the  "appropriate  action"  to  be  taken. 

As  mentioned  earlier,  benchmarking  can  be  extremely  beneficial  to  an  orga¬ 
nization.  It  allows  you  to  draw  upon  others'  successes  to  improve  your 
processes.  Benchmarking  involves  studying  other  similar  organizations  that 

Saform  above  expected  norms  and  applying  their  ideas  to  your  processes. 

on't  limit  yourself  to  studying  your  competition!,  Good  ideas  can  be 
transplanted  from  anywhere.  Benchmarking  is  taking  advantage  of  others' 
experience  to  avoid  pitfalls  encountered  earlier  somewhere  else.  More 
importantly,  it  allows  you  to  benefit  from  others'  "best  practices"  so  your 
organization  will  excel. 


d.  Communicating  Your  Intentions: 

Setting  the  stage  for  change  within  your  organization  can  be  as  important  as 
actually  making  the  changes,  Before  taking  any  action,  you  must  clearly 
explain  to  your  people  what  you  are  changing,  why  you  are  changing  it,  and 
when  it  will  happen. 

Communication  serves  two  purposes.  First,  it  educates  your  people.  Continu¬ 
ous  improvements  or,  in  a  few  cases,  new  processes  take  some  getting  used  to. 
The  time  lag  from  initial  introduction,  down  the  learning  curve,  to  final 
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integration  and  acceptance  can  be  minimized  by  education.  Second,  and  more 
importantly,  communication  shows  your  people  that  you  value  them  as  integral 
to  the  success  of  any  changed  processes  and  policies.  Explaining  your  actions 
will  maximize  your  chances  for  success  at  implementing  change.  The  degree 
to  which  you  explain  your  actions  up  front  will  serve  to  achieve  consensus 
earlier  than  otherwise  possible. 


e.  Take  Action: 

TAKE  ACTION!  This  may  not  be  easy.  A  continuous  improvement  philosophy 
requires  the  support  and  commitment  of  customers,  suppliers,  and  process 
owners  and  members.  Customer  dissatisfaction  and  process  problems  must 
now  be  seen  as  valuable  feedback  and  not  as  causes  for  punishment.  In  many 
cases,  reward  systems  will  require  changes  to  support  customer  and  process 
oriented  behavior. 

Be  careful  not  to  over-react  -  Deming  calls  this  "management  tampering."  It 
inevitably  leads  to  an  erratic  trend  and  an  increase  in  process  variation.  In 
follow-up  evaluations  of  the  implemented  change,  consider  whether  you  have 
addressed  a  symptom  and  not  the  root  cause  of  die  problem. 


f.  Repeat  the  Process: 

Even  now  you're  not  done!  Continuous  improvement  means  eternal  vigilance. 
You'll  need  to  continually  monitor  your  processes  to  determine  the  success  of 
process  changes.  The  steps  you've  just  taken  are  shown  in  the  flow-chart  ai 
Figure  6-1.  Use  them  as  a  guide  for  continually  taking  action  based  on  what 
your  metrics  tell  you. 


Figure  6-1  Metric  Implementation 


Remember  that  leadership  must  constantly  seek  feedback  from  both  internal 
and  external  sources.  The  process  of  quality  improvement  is  never  ending.  Y  ou 
must  continue  to  measure,  and  continue  to  act  on  what  your  metrics  tell  you,  and 
continue  to  update  your  metrics  to  facilitate  even  further  improvement. 
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Appendix  A:  Eight  Measurement  Tools 


Run  Chart . A-3 

Control  Chan . A-9 

Flow  Chan . A-21 

Cause  &  Effect  Diagram . A-23 

Check  Sheet . A- 25 

Pareto  Chan . A-27 

Histogram . A-31 

Scatter  Diagram . A-35 


Run  Chart 


What  it  Is: 

A  run  chan  is  a  graph  of  process  measurement  over  time.  It  can  be  used  to: 

*  Begin  to  understand  the  basic  characteristics,  including  variation,  of  e 
process. 

*  Document  variation  while  a  process  is  ongoing. 

•  Plot  historical  data  to  get  a  better  understanding  of  what  nas  happened  in  the 
past. 

•  Track  short  term  processes  where  each  resulting  product  or  service  is 
measured. 


How  to  use  It: 

•  Construct  the  Chart.  Label  the  vertical  axis  with  the  key  measurement  of  the 
process  being  measured.  Label  the  horizontal  axis  with  units  of  time. 

•  Plot  the  Data.  Collect  7-10  data  points  and  plot  them  on  the  chart.  Average 
all  the  points  and  plot  a  centerline. 

•  Interpret  the  Chart.  Interpret  the  chart  using  your  best  judgement.  Two 
common  tests  can  be  made  to  detect  significant  process  changes: 

-  A  run  of  seven  consecutive  points  on  one  side  of  the  average;  , 

-  Seven  or  more  consecutive  points  steadily  increasing  or  decreasing. 

•  Recompute  the  centerline  when  you  have  reason  to  believe  something  has 
changed  in  the  process.  If  you  do  recompute  the  centerline,  annotate  the 
change  on  the  chan. 


Points  to  Remember: 

•  For  ease  of  interpretation,  approximately  the  same  time  interval  should 
occur  between  measurements. 

♦  Not  every  variation  in  the  data  is  significant.  Since  the  factors  that  make  up 
a  process  vary  over  time,  any  measurment  corning  from  the  process  can 
reasonably  be  expected  to  vary.  Investigating  every  variation  as  though  it  is 
critical  can  be  frustrating  and  wasteful. 


1  Mirk  J.  Kiemele «rd Stephen R.  Schmidt.  Biiic  Statistic!:  Tooli  for  Corttinuour  Improvement.  Colorado  Sprtngt,  CO: 
Air  Acidemy  Prtis,  1990,  p.  2-18, 
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•  You  should  keep  in  mind  that  it  is  appropriate  to  start  a  run  chart  with  a 
limited  amount  of  data  or  a  limited  number  of  time  periods.  You  should, 
however,  consider  findings  from  this  intial  data  as  preliminary. 

Run  Chart  Example  1: 

AEDC  collected  statistics  on  data  deliverables  over  the  past  four  years  to 
determine  if  a  trend  was  forming. 


Fiscal  Year 

%  Of  Data 
Delivered  On  Time 

1987 

48% 

1988 

49% 

1989 

73% 

1990 

77% 

Partial  Year  1991 

94%' 

Simple  Average  68% 


Table  A»l:  AEDC  Data  Deliverables 
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This  is  displayed  on  a  run  chart  (Fig  A*  1).  It  should  be  noted  that  the  199 1  data 
is  not  for  a  full  year  and  should  be  considered  preliminary.  It  appears  that  there 
is  an  improvement  trend  in  this  process,  but  since  there  are  only  nve  data  points 
in  this  plot,  the  results  may  be  deceptive.  More  data  should  be  collected,  or  the 
data  collected  should  be  attributed  to  shorter  time  periods. 


—  On  Time  Average 


Figure  A-l:  Data  Deliverables  Run  Chart 


Run  Chert  Example  2: 

HSD  collected  data  over  a  20  month  period.  Samples  were  taken  every  other 
month,  the  following  data  resulted. 


Month  Sampled 

Median  Technical  Report 
Processing  Time  (In  Days) 

Jul 

2S0 

Sep 

260 

Nov 

258 

Jan 

224 

Mar 

34) 

May 

290 

Jul 

196 

Sep 

158 

Nov 

165 

Jan 

173 

Mar 

144 

Simple  Average  224  Day*  Of  Median  Days 


Table  A-2:  HSD  Technical  Report  Processing  Time 
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Jul  Sep  Nov  Jon  Mar  May  Jui  Sop  Nov  Jan  Mar 
Month! 


Madian  Days 
Avaragt 


Figure  A-2:  Technical  Report  Processing  Time  Run  Chart 


The  run  chart  shows  what  appears  to  be  a  favorable  trend.  This  apparent  trend 
should  be  analyzed  before  basing  any  decisions  on  this  information. 
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Control  Chart 


What  it  Is: 

A  control  chart  is  a  tool  for  analyzing  and  monitoring  the  variability  of  a 
process  over  time.  A  process  is  said  to  be  in  a  state  of  "statistical  control"  when 
the  variation  present  in  the  process  is  consistently  random  and  within  predictable 
limits.  When  a  process  is  in  this  state,  process  measurements  vary  randomly 
within  the  statistical  control  limits. 

Control  limits  and  tolerance  limits  are  not  the  same.  While  control  limits  are 
based  on  the  process  being  used,  tolerances  are  specified  by  drawings, 
specifications,  standards,  etc, 


A  control  chart  can  help  a  group  or  individual: 

•  Understand  and  recognize  variability  and  how  to  control  it. 

•  Identify  the  presence  of  "special  causes"  of  variation  or  changes  in 
performance  of  a  process,  i.e.,  a  shift  in  the  process  average,  change  in 
the  variance,  cyclic  behavior,  etc. 

•  Stop  trying  to  "fix"  a  process  that  is  varying  consistently  over  time  and 
objectively  determine  the  root  causes  of  problems. 


Caution: 

Control  charts  should  notbt  the/Irsttool  used  to  analyze  data.  Unless  you  have 
clearly  identified  the  process  in  question  and  have  data  to  support  investigating 
the  process  you  want  to  "control,"  your  improvement  efforts  will  not  be  fully 
successful. 


How  to  uaa  it: 

•  The  type  of  data  to  be  collected  will  dictate  the  type  of  control  chart  to  be 
used.  In  general,  variables  data  require  some  form  of  measurement,  e  g., 
length,  temperature,  etc.  and  the  recording  of  those  measurements. 
Attributes  data  require  a  good/bad  or  go/no-go  decision  and  counting, 
e  g.,  types  of  defects,  percent  late,  etc.  Since  attributes  data  doesn't 
require  detailed  measurements,  it  is  easier  and  less  costly  to  obtain.  Use 
the  Figure  A-3  to  determine  the  type  of  control  chan  needed. 

♦  Once  the  type  of  data  to  be  collected  is  determined,  follow  the  appropriate 
chart  construction  techniques  on  Pages  A-9  -  A- 14. 
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•  Eliminate  the  special  or  assignable  causes  of  variation.  The  existence  of 
these  causes  is  Indicated  by  one  or  more  of  the  following  seven  signals; 

-  One  or  more  points  outside  the  control  limits. 

-  Seven  or  more  points  on  one  side  of  the  centerline. 

•  Six  points  in  a  row  steadily  increasing  or  decreasing. 

•  Fourteen  points  alternating  up  and  down. 

-  Two  out  of  three  points  in  a  row  in  the  outer  third  of  the  control 
region. 

■  Fifteen  points  in  a  row  within  the  center  third  of  the  control  region. 

•  Eight  points  on  both  sides  of  the  center  line  with  none  in  the  center 
third  of  the  control  region. 

NOTE:  The  probability  that  any  of  these  seven  events  will  occur  at 
random  is  very  small,  which  is  why  they're  a  signal  that  something  has 
changed  in  your  process. 

«  When  you  have  eliminated  all  special  causes,  work  on  reducing  the 
remaining  variability  in  your  process.  This  is  usually  achieved  through 
fundamental  changes  to  the  process,  and  these  changes  require  manage¬ 
ment  assistance.  To  achieve  continuous  process  improvement,  it's  essen¬ 
tial  to  reduce  variability. 


*  Mm<  •  An  InMual  FtlunTi  MMAHn^t 
DMi  ■  A  Unt  Win  CMttM  On*  Or  Mm  CM  Mm 


Figure  A-3:  Control  Chart  Selection  Process 
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Construction  Steps: 

General: 


1 .  Determine  the  type  of  data  that  is  to  be  collected.  Consult  Figure  A-  3  to 
determine  the  appropriate  type  of  control  chan  to  employ. 

2.  Collect  the  data. 

3 .  Mark  the  vertical  (y)  axis  with  the  name  of  the  variable  measured  and  its 
unit  of  measurement. 

4.  Mark  the  horizontal  (x)  axis  with  units  of  time. 

;5.  Label  the  chan  with  descriptive  title(s)  which  describe  the  measurement 
and  its  time  frame. 

6.  Determine  sub-groups,  calculate  upper  and  lower  control  limits,  and 
mean  as  appropriate. 

7.  Divide  and  label  the  horizontal  and  vertical  axes  into  even  increments 
which  easily  allow  all  the  data  to  be  plotted. 

8.  Plot  the  data  points  and  the  control  limits. 


Chart  Construction  Techniques: 

X  and  n  Control  Chart: 


1.  You  will  analyze  the  ranges  (R)  chart  first,  then  the  sample 
averages  (X). 

2.  Compute  R  for  each  subgroup,  R  ■  Xmix  - 

3.  Scale  the  lower  portion  of  the  chart  and  plot  the  points. 

4.  Compute "R  ■  (Rt+  Rj  +  +  R3)/k  «■ _ 


5.  Select  A2,  D3  and  D4  from  the  table  below  based  on  your  subgroup  size. 


Subgroup 

SlZfi 

A, 

^3 

n. 

Subgroup 

At 

% 

a. 

2 

1.880 

0 

3,267 

6 

0.483 

0 

2.004 

3 

1.02? 

0 

2.574 

7 

0.419 

0.076 

1.924 

4 

0.729 

0 

2.282 

8 

0.373 

0.136 

1.864 

5 

0.577 

0 

2.114 

9 

0.337 

0.184 

1.816 

10 

0.308 

0.223 

1.777 

Table  A>3;  Control  Chart  Constants 

6.  Compute  UCLR  **  D4  x  R  = _ x 


7.  Compute  LCLR  »  D3  x  R  ■ _ x _ = _ 

8.  Draw  the  control  lines  and  check  for  any  points  out-of-control.  Recom¬ 
pute  the  control  limits,  if  necessary. 

9.  Compute  5?  for  each  subgroup. 

10.  Scale  the  upper  portion  of  the  chan  and  plot  the  points. 

1 1 .  Compute  ^  m  (Xs  +  +  •••  +  X^/k  ■ _ 

12.  Compute  UCL_  *  5?  +  (A,  x  R) » _ +  { _ x _ ) » _ 

!  3.  Compute  LCL_  =  X  -  (A2  x  R)  = _ •  ( _ x _ )  = _ 

14.  Draw  the  control  lines  and  check  for  any  points  out  of  control.  Recom¬ 
pute  the  control  lines  on  both  charts,  if  necessary. 
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X  find  MR  Control  Chart: 


1.  You  will  analyze  the  moving  range  (MR)  chart  first*  then  the 
individuals  (X). 

2.  Compute  MR  for  each  subgroup.  MR  is  the  absolute  value  of  the  differ¬ 
ence  between  consecutive  range  values.  MR  ■  R  -  Rn  ,  where  n  and  n- 
1  are  consecutive  observations.  Remember  there  is  no  MR  associated 
with  the  first  X  value. 

3.  Scale  the  lower  portion  of  the  chart  and  plot  the  MR  points.  Remember 
to  start  at  the  second  position  along  the  time  axis, 

4.  Compute  MR  *  (MRj+  MR2  +  +  MRk  j)/(k-l)  ■ _ 

5.  Compute  UCLMR  -  3.268  x  MR  «  3.268  x _ * _ 

Note:  LCLmr  ■>  0 

6.  Draw  the  control  lines  and  check  for  any  points  out  -of-control.  If  a 
point  is  out-of-control  investigate  the  associated  X  value.  If  the  X  value 
is  removed  as  abnormal,  remember  that  one  MR  will  be  removed  and 
another  will  change.  Recompute  die  control  limits,  if  necessary. 

7.  Scale  the  upper  portion  of  the  chart  and  plot  the  X  points. 

8.  Compute  X  ■  (Xj  +  X2  +  »*•  +  Xk)/k  ■ _ 

9.  Compute  UCLx  -X  +  (3  x  (MR/  1.128)) 

« _ +  (3  x  ( _ / 1.128))  _ 

10.  Compute  LCL^  -  X  +  (3  x  (MR/  1. 128)) 

» _ +  (3  x  ( _ / 1.128))  - _ 

11.  Draw  the  control  lines  and  check  for  any  points  out-of-control.  Recom¬ 
pute  the  control  lines  on  both  charts,  if  necessary. 


A-13 


Attributes  Control  Charts: 


»  Determine  which  chart  to  use.  The  information  in  Table  A-5  helps  clarify 
the  options  in  the  Figure  A-3. 

Defect:  An  individual  failure  to  meet  a  single  requirement. 

Defective:  A  unit  containing  one  or  more  defects. 


Varying  Sample  Size 

Constant  Sample  Size 

Defectives 

p-  Chart 

Used  to  chart  the  function 
or  percent  defective 

np-  Chart 

Used  to  chart  the  number 
of  defects  In  a  subgroup 

Defects 

U-  Chan 

Used  to  chan  the  number 
of  defects  per  unit. 

C-  Chart 

Used  to  chart  the  number 
of  defects  in  a  subgroup 

Table  A-4:  Attributes  Control  Chart  Selection  Grid 


•  Collect  10- 20  subgroups  of  data.  Each  subgroup  consists  of  multiple  data 
points  that  are  arranged  in  a  rational  manner,  day,  lot,  office,  etc,  The  size 
of  each  subgroup  is  represented  by  "n." 

•  Use  the  general  construction  steps  from  page  A-9  with  the  following 
formulas: 
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p-Chart: 


n*  size  of  subgroup 


■  fraction  or  percent  defective 


p*  total  defective  -  centerline  ■  average  fraction  defective 
total  inspected 

UCL  »p  +  3*  *\/1Pn  -v'm  upper  control  limit  (varies  by  subgroup) 

■  lowercontrol  limit  (varies  by  subgroup) 


LCL»p  +  3* V 


Note:  You  can  use  an  average  subgroup  size  to  obtain  a  single  set  of  control 
limits  if: 

•  The  largest  subgroup  size  is  less  than  twice  the  average  subgroup  size. 

•  The  smallest  subgroup  size  is  more  than  half  the  average  subgroup  size. 

Plot  p  for  each  subgroup. 


rip  Chart: 

p,  n=  same  as  for  p-chart,  except  n  must  be  constant, 
np*  centerline  «  average  number  of  defectives 
UCL»np+3*  ynp(i-p) 

LCL  -  np  -  3  * 
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u  Chart: 


Plot  u  for  each  subgroup 


C  Chart: 

C*  number  of  defects  per  subgroup 


total  defects 


total  number  of  suggroups 
UCL-C  +  3*  VT 


- a  centerline 


LCL  -  C  -  3  * 


VT 


Plot  C  for  each  subgroup 
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Control  Chart  -  Example  1: 

X  and  R  Chart 


X  =  4.37 
R  =  0.34 
n  =  5 

A  2=  0.577 
Dg=0 
D4=  2.114 

Table  A-5:  X-R  Chart  Data 


WMk 

Mon. 

1 

Tuoo 

WOO. 

Thuro. 

Frl 

X 

R 

1 

4J0 

460 

4  JO 

4.40 

4.40 

4.46 

0.2 

2 

4.GC 

4.80 

4.40 

4.30 

4.10 

4.36 

0.8 

3 

4.60 

4.10 

4.40 

4.40 

4.10 

4.32 

0.8 

4 

4.40 

4.30 

4.40 

4.20 

4.30 

4  32 

0.2 

5 

4.30 

4.30 

4.40 

4.20 

4.30 

4.30 

0.2 

e 

4.60 

4.60 

4.20 

4.50 

4.40 

4.46 

0.4 

7 

4.10 

4.30 

4.60 

4.80 

4.20 

4.34 

0.5 

a 

4.80 

4.80 

4.40 

4.60 

4.40 

4.40 

0.2 

9 

4.40 

4.20 

4.60 

4.60 

4.20 

4.40 

0.4 

10 

4.20 

4.20 

4.20 

4.50 

4.20 

4.26 

■9 

$  Fw  Lunch 


UCL  .  4  57 


X  •  4.37 


LCL  »  4.15 
Wok 


UCL  -  .72 


R.  34 

LCL  -  0 


Figure  A-4:  X  and  R  Control  Chart  $  For  Lunch 


Control  Chart  Example  3: 

The  following  is  an  example  from  SSD  in  w'hich  several  types  of  control  charts 
were  used  to  analyze  a  Service  Report  process. 


Control  Charts 


Samnle  Month 

X 

R 

_iL_ 

1 

143 

244 

0.71 

2 

139 

348 

0.71 

3 

1)6 

337 

0.54 

4 

84 

9 

0.54 

5 

88 

155 

0.52 

6 

60 

133 

0.54 

7 

90 

154 

0.60 

8 

210 

294 

0.51 

i 

9 

125 

241 

0.74 

10 

51 

29 

0.47 

11 

56 

64 

0.62 

12 

65 

56 

0.39 

13 

143 

351 

0.43 

14 

— 

— 

0.65 

15 

— 

— 

0.70 

Central  Line 

105 

186 

0.58 

17CL 

212 

393 

0.76 

LCL 

0 

0 

0.40 

Table  A-7:  Service  Reports  Control  Chart  Data 
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"X-DAYS  TO  CLOSE 


Figure  A-6:  Service  Reports  Time  To  Closed  Chart 


R-DAYS  TO  CLOSE 


Figure  A-7:  Time  To  CIose>p  Chart 


Figure  A-8:  SR  Fraction  Of  Open  Reports  Overage  p-Chart 
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Flow  Chart 


What  It  It: 

A  flow  chart  is  a  graphic  representation  of  all  the  major  steps  of  a  process.  It 
can  be  used  to  help: 

•  Understand  a  complete  process. 

•  Identify  the  cridcal  stages  of  a  process. 

•  Locate  problem  areas. 

•  Show  relationships  between  different  steps  in  a  process, 


Th to  Symbol. 

Royroooitu... 

lotiu  EumptM  Ar*~ 

o 

Sun/Siop  , 

Rocloyo  Diroctlon  Prom  Beu 
Submit  Plnol  R  opart 

Wilt*  Up 

Oo  To  Sloop 

o 

Doclilon  Point 

Approvo/Dlupp.  ovo 
Aosopt/Rojocl 

Vm/No 

Puo/FiU 

AoUvIty 

Fill  Out  Tmvol  Vouchor 

WolJi  Down  ToCBPO 

Bruoh  Tooth 

Oct  Drootod 

Table  A-8:  Flow  Chart  Symbols 

How  to  usa  it: 

•  Define  the  start  point  and  finish  point  for  the  process  to  be  examined. 

•  Describe  the  current  process,  charting  the  whole  process  from  beginning 
to  end.  You  can  use  the  symbols  shown  in  the  figure  to  improve  the  clarity 
of  your  flow  chart,  but  don't  get  hung  up  on  symbols. 

•  Chart  the  ideal  process  and  try  to  identify  the  easiest  and  most  efficient 
way  to  go  from  the  "start  block"  to  the  "finish  block."  While  this  step  isn't 
absolutely  necessary,  it  does  make  it  easier  to  do  the  next  step... 

•  Search  for  improvement  opportunities.  Identify  all  the  areas  that  hinder 
your  process  or  add  little  or  no  value.  If  you  did  the  previous,  optional 
step,  examine  all  areas  that  differ  from  your  ideal  process  and  question 
why  they  exist. 
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•  Build  a  new  flow  chart  that  corrects  the  problems  you  identified  in  the 
previous  step. 


Helpful  Hint: 


Consider  putting  the  steps  of  your  process  on  index  cards  or  stick-back  note 
paper.  This  lets  you  rearrange  the  diagram  without  erasing  and  redrawing. 
Making  it  easy  to  reconfigure  the  diagram  encourages  contributions. 


Flow  chart  axampla: 

The  following  flow  chart  was  built  to  show  the  administrative  process  for  an 
Air  Force  Form  9. 


Figure  A-9;  Flow  Chart  For  AF  Form  9  Process 
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Cause  &  Effect  Diagram 


What  It  la: 

A  cause  and  effect  diagram  graphically  illustrates  the  relationship  between  a 
given  outcome  and  all  the  factors  that  influence  this  outcome.  It  is  sometimes 
called  an  Ishikawa  Diagram  or  a  Fishbone  Diagram.  Cause  and  effect  diagrams 
provide  a  structured  approach  to  determining  the  root  causes  of  a  problem,  an 
objective,  or  some  other  effect.  They  can  be  used  to: 

•  Determine  the  factors  that  cause  a  positive  or  negative  outcome  or  effect. 

•  Focus  on  a  specific  issue  without  resorting  to  complaints  and  irrelevant 
discussion. 

•  Identify  areas  where  there  is  a  lack  of  data. 


Howto  uto  It: 

•  Specify  the  effect  to  be  analyzed.  The  effect  can  be  positive,  e.g. 
objectives, or  negative,  e.g.  problems.  Place  it  in  a  box  on  the  right  side 
of  the  diagram. 

•  List  the  mqjor  categories  of  the  factors  that  influence  the  effects  being 
studied.  The  "4  Ms"  (methods,  manpower,  material,  machinery)  or  the 
"4Ps" (policies,  procedures,  people,  plant)  are  commonly  used  as  a 
starting  point. 

•  Use  an  idea-generating  technique  to  identify  the  factors  and  subfactors 
within  each  major  category.  An  easy  way  to  begin  is  to  use  the  major 
categories  as  a  catalyst.  For  example,  "What  policies  are  causing. . .?" 

•  Look  for  factors  that  appear  repeatedly  or  are  a  root  cause  to  the  effect 
and  list  them. 

•  Prioritize  your  list  of  causes  using  a  decision-making  tool.  Keep  in  mind 
that  the  location  of  a  cause  in  your  diagram  is  not  an  indicator  of  its  im¬ 
portance.  A  subfactor  may  be  the  root  cause  to  all  of  your  problems. 


Holpful  Hint: 

Consider  using  an  objective  instead  of  a  problem  as  the  effect  to  be  discussed. 
A  problem  oriented  focus  may  produce  "finger  pointing,"  while  focusing  on 
desired  outcomes  foster  pride  and  ownership  over  productive  areas.  The 
resulting  positive  atmosphere  will  enhance  the  group  s  creativity. 
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Figure  A- 10:  Error  Free  Documents  Cause  and  Effect 


Check  Sheet 


What  Yi  It: 

A  check  sheet  is  a  simple  form  which  can  be  used  to  collect  data  in  an  organized 
manner  and  easily  convert  it  into  readily  useful  information.  With  a  check 
sheet,  you  can: 

•  Collect  data  with  minimal  effort. 

•  Convert  raw  data  into  useful  information. 

•  Translate  opinions  of  what  is  happening  into  what  is  actually  happening. 
In  other  words,  "I  think  the  problem  is. . becomes  "The  data  says  the 
problem  is..." 


How  to  uao  It: 

•  Clearly  identify  what  is  being  observed.  The  events  being  observed 
should  be  clearly  labeled.  Everyone  has  to  be  looking  for  the  same  thing. 

•  Keep  the  data  collection  process  as  easy  as  possible.  Collecting  data 
should  not  become  a  job  in  and  of  itself.  Simple  check  marks  are  easiest. 

•  Group  the  collected  data  in  a  way  that  makes  the  data  valuable  and 
reliable.  Similar  problems  must  be  in  similar  groups. 

•  Try  to  create  a  format  that  will  give  you  the  most  information  with  the 
least  amount  of  effort. 
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Check  Sheet  Example: 

Table  A-9presents  data  collected  or  Unfunded  Requirement  (UR)  processing 
times  in  FY91.  The  check  sheet  shown  in  Table  A-10  was  built  to  better 
understand  this  data. 


Days  To  Processes  URs 


OBS 

Days 

OBS 

Days 

OBS 

Days 

1 

28 

8 

19 

11 

9 

2 

18 

7 

15 

12 

12 

3 

38 

8 

119 

13 

117 

4 

9 

9 

97 

14 

114 

5 

79 

10 

78 

IS 

8 

Table  A*9:  UR  Processing  Time  Data 


Class  (Days) 

Observations 

Total 

0-9 

II! 

3 

10-19 

1111 

4 

20-29 

1 

1. 

30-39 

0 

40-49 

0 

50-59 

1 

1 

60-69 

0 

70-79 

1 

2 

80-89 

0 

90-99 

1 

1 

100-109 

c 

110-119 

III 

3 

Total  15 

Table  A-10:  UR  Processing  Time  Check  Sheet 
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Pareto  Chart 


What  It  la: 

A  Pareto  chart  is  a  bar  chan  used  to  separate  the  "vital  few"  from  the  "trivial 
maiiy."  These  charts  are  based  on  the  Pareto  Principle  which  states  that  10- 
20  percent  of  the  problems  have  80-90  percent  of  the  impact.  These  10-20 
percent  of  the  problems  are  the  "vital  few"  and  the  remaining  problems  are  the 
"trivial  many."  Pareto  chans  help  groups  and  individuals: 

•  Separate  the  few  major  problems  from  the  many  possible  problems  so 
that  improvement  efforts  can  be  properly  focused. 

•  Categorize  and  prioritize  problems  and/or  data. 

«  Determine  which  problems  that  are  most  imponant  using  data  and  not 
intuition  or  perception. 


How  to  use  It: 

•  Use  idea-generation  techniques  to  list  all  the  possible  problems  in  a 
particular  process. 

•  Use  existing  reports  or  collect  new  data  on  the  process,  Be  sure  these 
utiits  are  consistent  throughout  your  data. 

•  Label  the  units  of  measure  on  the  left  venical  axis  and  the  categories 
of  problems  on  the  horizontal  axis. 

•  Order  the  categories  according  to  their  frequency,  how  many,  not 
their  classification,  what  kind.  Use  a  descending  order  from  left  to 
right 

•  Check  that  the  labels  are  clear  and  identify  the  measurement  units 
used. 


Points  to  Rtmsmbsr: 


•  The  measurement  units  used  can  significantly  impact  your  Pareto  chart. 
For  example,  100  cosmetic-type  defects  may  account  for  only  a  fraction 
of  the  total  cost,  while  2  material-type  defects  may  account  for  a  large 
percentage  of  the  cost.  You  must  determine  if  cost  or  number  of  defects 
is  most  important. 

•  It  is  essential  to  use  the  same  units  of  measure,  and  clearly  mark  these 
units  on  the  chart, 


Nested  Pareto  Charts: 

Broad  causes  can  be  broken  down  into  more  specific  areas  to  facilitate 
improvement  efforts.  These  specific  areas  are  "nested"  within  the  broad  causes, 
hence  the  term  "Nested  Pareto  Charts.  Figure  A- 14  illustrates  the  concept  of 
nested  Pareto  charts. 


Stratlflcetion  Analysis: 

The  same  data  can  be  plotted  against  different  potential  causes  to  determine  the 
significant  problem.  This  concept  is  very  useful  when  your  original  Pareto 
chart  doesn’t  clearly  identify  one  or  two  significant  problem  areas.  Use  your 
imagination  and  be  creative  when  you  define  your  problem  categories.  Figures 
A- 12,  and  A- 13  show  an  example  of  stratification  analysis. 


Pareto  Chart  -  Example  1 : 

An  investigation  of  Laser  Photography  reliability  was  undertaken  at  AEDC 
because  the  existing  84%  reliability  did  not  meet  customer  requirements. 

A  sample  of  recent  test  shots  showed  36  failures.  These  occurred  in  three 
categories:  trigger  failures,  shot  failures,  and  system  failures.  23  of  the  36  were 

trigger  failures,  7  were  shot 
failures,  the  remaining  6 
were  classified  as  system 
failures.  A  simple  Pareto 
Chart  (Figure  A- 11)  clari¬ 
fies  the  area  to  investigate 
further.  These  failures  were 
further  investigated  In  Fig¬ 
ures  A- 12  and  A- 1 3.  These 
are  examples  of  stratifica¬ 
tion. 

This  data  identified  trigger 
stations  24  and  8  as  the  main 
culprits  of  trigger  failures. 
This  led  to  a  closer  look  at 
these  stations.  Marginal 
components  at  these  stations 
were  replaced  and  the  trigger  sensitivities  were  optimized.  The  data  collection 
effort  is  continuing  to  determine  if  the  problems  located  and  addressed  will 
result  in  improved  performance. 


Ftlun  Ctogiyy 

Figure  A-llt  Laser  Photography  Failures 
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Figure  A-12:  Trigger  Failures  Figure  A-13:  Laser  Station  Identification 
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Pareto  Chart  Example  2 

Nested  Parsto  Charts 
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Figure  A-14:  Defects  Analysis 
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Histogram 


What  It  Is: 

A  histogram  is  a  ba*  graph  used  to  depict  the  variability  of  a  data  set  in  the  form 
of  a  frequency  distribution.  A  histogram  can  help  a  group  or  individual: 

•  Display  the  variability  in  a  data  measurement. 

*  Quickly  and  easily  assess  the  shape  of  the  underlying  distribution  of  a 
process. 


How  to  use  it: 

•  Collect  as  many  measurable  data  points,  i.e.,  time,  length,  speed...,  as 
possible. 

•  Count  the  total  number  of  points  you  have  collected. 

•  Subtract  the  smallest  value  in  the  dataset  from  the  largest.  Tms  value  is 
the  range  of  your  dataset. 

•  Divide  the  range  into  a  certain  number  of  intervals,  or  bars,  on  the  graph. 
Use  the  following  guidelines  to  determine  the  appropriate  number  of 
intervals  to  use. 


Under  50 

5-7 

50-99 

6-10 

100-249 

7-12 

Over  249 

10-20 

•  Divide  the  range  by  the  number  of  intervals.  Round  your  answers  up  to 
a  convenient  value.  For  example,  if  the  range  of  the  data  is  17  and  you 
have  decided  to  use  9  intervals,  then  your  interval  width  is  1 .88.  This  can 
be  rounded  to  1.9  or  2.0.  It  is  helpful  to  have  intervals  defined  to  one  more 
decimal  place  than  the  data  collected. 

♦  Determine  the  starting  point  of  each  interval.  Start  with  the  smallest  data 
point  as  the  starting  point  of  the  first  interval.  Now  add  the  interval  width 
that  you  determined  in  the  previous  step.  The  sum  is  the  starting  point  for 
the  next  interval.  For  example,  if  your  smallest  data  point  is  10,  the  next 
is  12  and  so  on.  Label  these  intervals  along  the  horizontal  axis. 


•  Count  the  number  of  data  points  that  fall  within  each  interval  and  plot  this 
frequency  on  the  histogram.  Keep  in  mind  that  each  datapoint  can  appear 
in  only  cne  interval.  For  example,  if  your  first  interval  begins  with  10.0  and 
the  second  with  12.0,  then  all  data  points  that  are  equal  to  or  greater  than 
10.0  and  still  less  than  12.0  are  counted  in  the  fust  interval. 


Points  to  Rsmembsr: 

•  Each  data  point  appears  on  one  and  only  one  interval. 

•  The  number  of  intervals  can  influence  the  pattern  your  data  will  take. 

•  Don't  expect  the  histogram  to  be  a  perfect  picture;  variations  v/ill  occur. 
Ask  yourself  if  the  picture  is  reasonable  and  logically  correct. 


Histogram  -  Exiimpla: 

Table  A- 11  is  a  check  sheet  that  was  built  from  data  collected  on  unfunded 
requirements  processed  by  HQ  AFSC  in  FY  90.  This  check  sheet  was  used  to 
build  Figure  A*  IS,  a  histogram  that  was  used  to  display  the  distribution  of  UR 
processing  times. 


Processing  Time 
(In  Days) 

Number  Of  Observations 

Total 

Range 

Mid-Point 

0- 19 

MMMMII 

22 

10 

20-39 

MMM  MM 

25 

30 

40-59 

MMM  till 

19 

50 

60-79 

MM  II 

12 

70 

80-99 

M  INI 

9 

90 

100-119 

llll 

4 

110 

120-139 

Ill 

3 

130 

140-159 

III 

3 

150 

160-179 

0 

170 

180-199 

llll 

4 

190 

200-360 

Mil 

7 

240* 

*  Computed  From  Del*  Not  Shown  Here  1  uo 

Table  A-ll:  UR  Processing  Time  Check  Sheet 


10  30  SO  70  00  110  130  'SO  170  100  240 

Elapsed  Time  In  Days 


Figure  A-15:  UR  Processing  Time  Histogram 
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Scatter  Diagram 


What  It  is: 

A  scatter  diagram  is  a  graph  used  to  reveal  the  possible  relationship  between 
two  variables.  It  can  help  a  group  or  individual: 

*  Identify  possible  causes  of  problems. 

•  Recognize  the  correlation  between  one  important  variable  and  another. 


How  to  U89  it: 

•  Collect  the  data  in  pairs.  For  each  data  point  obtained  for  the  first 
variable,  a  data  point  must  be  available  for  the  second  variable  which 
represents  the  same  time,  location,  or  occasion.  The  larger  the  number  of 
data  points  collected,  the  more  reliable  the  conclusions  drawn  from  the 
scatter  diagram. 

•  Mark  the  horizontal  axis  with  the  name  and  units  of  measure  of  the 
variable  suspected  as  being  the  causal  or  independent  variable. 

•  Mark  the  vertical  axis  with  the  name  and  units  of  measure  of  the  variable 
suspected  as  being  the  effect  or  dependent  variable. 

•  Label  the  chart  with  descriptive  titles  and  the  appropriate  time  frame. 
Divide  and  label  both  axes  into  increments  which  allow  all  data  to  be 
plotted. 


•  Plot  the  data.  As  you  plot  each  point  circle  repeated  points.  Look  for 
patterns  in  the  data,  Figure  A- 16  will  help  you  interpret  patterns  in  the 
plot 


x  x 


Poiltlv*  Correlation  Negative  Correlation 

An  IwcfiiM  In  X  Miv  An  InerooM  In  X  Miv 

CauM  An  incmua  In  V  Oiuh  An  Dtcriu*  In  V 


X 

No  Correlation 
V  Do««  Net  Appear  To  Dopond  On  X 

Figure  A- 16:  Scatter  Diagrams 

Points  to  Romsmber: 

•  This  technique  only  shows  that  a  relationship  exists,  not  that  one  variable 
causes  the  other.  In  general,  it  is  wise  to  not  base  conclusions  about  cause- 
and-effect  relationships  solely  on  the  basis  of  a  scatter  diagram. 

•  As  with  all  statistical  tools,  it  is  better  to  have  more  data  than  not  enough . 
Scatter  diagrams  based  on  small  sample  sizes  can  be  misleading, 
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Scatter  Diagram  Example: 


This  example  show  s  actual  data  from  a  manpower  study .  The  measurement  was 
taken  to  determine  if  areladonship  for  predicted  workload  could  be  established. 


Work  Center:  Base  Supply  Funds  Management 
Workload  Factor  (WLF):  An  Item  Record  Loaded 

WLF  Definition:  The  Average  Monthly  Number  Of  Item  Records  For 

Base  Supply  For  Which  Supply  Inventory  Section 

Performs  Inventory 

Location 

Manhoura  (Y) 

Avg.  Workload*  fXl 

Aviano 

258.08 

41.65 

Bcntwaters 

323.81 

65.64 

Hahn 

301.45 

73.74 

Lackland 

279.56 

16.86 

Lowry 

350.80 

42.76 

Williams 

222.75 

40.04 

Wright-Patterson 

529.35 

130.37 

Dover 

292.08 

55.16 

Little  Rock 

355.09 

90.56 

Hickam 

515.46 

95.52 

Kadena 

496.79 

127.79 

Carswell 

265.24 

72.50 

Fairchild 

405,44 

72.43 

Malm  strom 

299.52 

64.90 

Pease 

357.16 

54.21 

Whiteman 

247.47 

29.17 

Edwards 

438.11 

101.75 

Mt.  Home 

439.24 

54.56 

Myrtle  Beach 

*  In  Thousand! 

340.76 

40.23 

Table  A-12:  Workload  Factor  Study  Data 
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When  the  measurement  data  in  Table  A- 1 3  was  plotted  on  Figure  A- 1 3  a  strong 
positive  correlation  between  manhours  required  and  item  records  loaded  was 
identified.  This  informadon  was  then  regressed*  and  the  results  became  the 
basis  for  manning  levels  at  base  supply  shops. 


Workload 

(In  Thousand*) 


Figure  A*  17:  WorkLoa'4  Fader  Study  Scatter  Diagram 


Appendix  B  Glossary 


Activity  •  Segments  of  processes  that,  when  coordinated,  add  up  to  a  particular 
process. 

Benchmarking  •  A  reference  point  established  by  the  “lead”  organization 
within  an  "Industry"  that  can  be  used  as  a  comparison  point  for  other 
organizations  in  the  same  "Industry." 

Best  practices  •  Closely  related  to  benchmarks,  these  are  the  state-of-the-art 
process  execution  practices. 

Big  Picture  Metric  (BPM)  •  A  top  level  measure  of  the  health  of  the  command 
and/or  its  critical  processes.  It  should  have  a  direct  link  to  the  Command's 
Coals.  A  BPM  will  usually  be  supported  by  one  or  more  command  standard 
metrics.  There  will  be  a  select  few  BPMs,  designated  by  the  Command 
Leadership. 

Cause  •  A  proven  reason  for  the  existence  of  a  defect. 

Cauae  and  effect  diagrams  •  A  diagram  that  graphically  illustrates  the  rela¬ 
tionship  between  a  given  outcome  and  all  the  factors  that  influence  this 
outcome.  Also  called  Ishikawa  or  Fishbone  Diagram. 

Check  Sheet  •  A  simple  form  used  to  collect  data  in  an  organized  manner 

Command  Standard  Metric  (CSM)  •  A  metric  for  which  a  common  approach 
for  measuring,  reporting  and  presenting  has  been  agreed  to  by  the  Field 
Commanders. 

Command  standard  presentation  format  •  A  standard  for  graphically  pre¬ 
senting  data.  A  command  standard  for  chart  format.  A  format  should  not  be 
confused  with  the  metric  itself. 

Common  cause  •  A  source  of  variation  that  is  always  present;  part  of  the 
random  variation  inherent  in  the  process  itself. 

Control  charts  •  A  tool  used  to  analyze  process  variability  over  time.  They 
measure  the  process  in  a  time  dimension  and  show  movement  toward  or  away 
from  an  average.  Control  charts  have  statistically  calculated  upper  and  lower 
control  limits. 

Control  limits  (Upper  &  Lower)  •  Statistically  derived  limits  that  represent 
upper  and  lower  bounds  for  processes  that  are  considered  “in  control. 

Customer  •  An  organization  or  individual  to  whom  you  provide  a  product  or 
service. 
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Detection  •  A  reactive  quality  assurance  strategy  that  attempts  to  identify 
unacceptable  output  after  it  has  been  produced  ana  separate  it  from  the  good 
output  Also  known  as  inspection. 

Fishbone  diagram  •  See  "Cause  and  Effect  Diagram." 

Flow  chart  •  A  graphic,  structured  representation  of  all  the  major  steps  in  a 
process. 

"4Ms"  •  Major  categories  often  used  to  build  a  Cause  and  Effect  Diagram. 
Manpower,  Machines,  Methods,  Materials. 

"4Ps”  •  Major  categories  often  used  to  build  a  Cause  and  Effect  Diagram. 
People,  Procedures,  Policies,  Plant. 

Goal  •  A  statement  describing  a  desired  future  condition  or  change  without 
being  specific  about  how  much  and  when. 

Histogram  •  A  bar  chart  used  to  depict  the  average  and  variability  of  a  data  set. 

Ishikawa  diagram  •  See  "Cause  and  Effect  Diagram" 

Metric  •  A  measurement,  taken  over  a  period  of  time,  that  communicates  vital 
information  about  a  process  or  activity  .  A  metric  should  drive  appropriate 
leadership/management  action.  Physically,  a  metric  package  consists  of  three 
parts:  (1)  An  operational  definition,  (2)  Measurement  over  time,  (3)  A 
presentation  package. 

Metric  classes  •  General  division  for  types  of  metrics  within  the  command. 
These  classes  are  broadly  related  to  command  mission  areas.  There  are 
presently  5  classes  identified:  Program  Management,  Test  and  Evaluation 
(T&E),  Science  and  Technology  (S&T),  Base  Operating  Support  (BOS),  and 
Functional.  These  classes  may  change. 

Metric  package  *  Three  elements  that  physically  make  up  a  metric.  (1)  An 
operational  definition,  (2)  Measurement  and  data  recording,  and  (3)  Metric 
presentation  package. 

Metric  presentation  package  •  A  two-part  package  needed  to  communicate 
a  metric.  This  package  consists  of  a  description  of  the  metric  and  a  graphical 
illustration.  The  graphical  illustration  can  be  of  many  types  but  usually  should 
be  a  run  chart  or  control  chart 

Mission  *  A  single  overriding  goal  statement  for  an  organization.  It  should 
encompass  all  of  an  organization's  significant  activities. 
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Objective  >  A  more  specific  statement  describing  a  desired  future  condition  or 
change  than  a  goal,  it  includes  measurable  end  results  to  be  accomplished 
within  specific  time  limits. 

Operational  Definition  *  A  detailed,  unambiguous  definition  that  provides 
enough  information  to  allow  consistent,  repeatable  and  valid  measurement. 

Pareto  Chart  •  A  type  of  bar  chart  used  to  separate  the  "vital  few"  from  the 
"trivial  many."  Based  on  the  Pareto  Principal  which  states  that  10-20  percent 
of  the  problems  have  80-90  percent  of  the  impact. 

PDSA  •  (Plan,  Do,  Study,  Act)  cycle-  A  four  step  variation  of  the  scientific 
method  for  managing  process  improvement.  PDSA  can  be  described  as  applied 
research  where  the  emphasis  is  on  getting  things  to  work  well  without  being 
overly  concerned  with  the  theoretical  foundation.  It  encourages  practical 
problem  solving  by  people  who  are  knowledgeable  about  a  process  (also 
known  as  Shewhart  cycle). 

Prevention  •  A  proactive  quality  assurance  strategy  that  improves  quality  by 
directing  analysis  and  action  towards  correcting  the  production  process. 

Process  •  Any  specific  combination  of  machines,  tools,  methods,  materials, 
and/or  people  employed  to  attain  specific  characteristics  in  a  product  or  service. 
A  change  in  any  of  the  constituents  results  in  a  new  process. 

Process  capability  •  The  measured,  built-in  reproducibility  of  the  product 
turned  out  by  the  process.  Such  a  determination  is  made  using  statistical 
methods,  not  wishful  thinking.  The  statistically  determined  pattern  or  distribu¬ 
tion  can  only  then  be  compared  to  specification  limits  to  decide  if  a  process  can 
consistently  deliver  the  product  within  those  parameters. 

Product  •  Output  of  a  process.  This  may  be  goods  or  services. 

Run  Charts  •  A  graph  of  a  process  measurement  over  time. 

Quality  *  All  features  and  characteristics  of  a  product  or  service  that  bear  on 
its  ability  to  satisfy  stated  or  implied  customer  needs. 

Range  •  The  difference  between  the  smallest  value  and  the  largest  value  in  a 
dataset 

Service  •  Work  performed  for  someone  else. 

Scatter  diagram  •  A  type  of  graph  used  to  reveal  the  possible  relationship 
between  two  variables. 
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Special  cause « A  source  of  variation  that  is  not  always  part  of  the  process,  but 
arises  because  of  specific  circumstances;  also  called  an  assignable  cause. 

Stable  process  •  A  process  in  statistical  control. 

Statistical  control  •  The  condition  describing  a  process  when  the  variation 
present  in  that  process  is  consistently  random  and  predictable  over  time. 

Statistical  process  control  •  The  application  of  statistical  techniques  for 
measuring  and  analyzing  the  variation  in  processes. 

Subgroup  *  One  or  more  data  points  rationally  grouped.  Subgroups  are  used 
in  control  charts. 

System  *  An  interdependent  group  of  items,  people,  or  processes  with  a 
common  purpose. 

Task  *  Specific  activity  necessary  in  the  function  of  an  organization. 

TQM  •  Total  Quality  Management  A  leadership  philosophy,  organizational 
structure,  and  working  environment  that  fosters  and  nourishes  a  personal 
accountability  and  responsibility  for  quality  and  a  quest  for  continuous 
improvement  in  products,  services,  and  processes. 

Trend  •  A  measurement  over  time  extending  in  a  general  direction. 

User  •  An  external  organization  that  uses  the  final  product  or  service. 

Variation  *  The  difference  among  individual  outputs  of  the  same  process. 
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